New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Llama65B patch for int4 fp32 #1769

Draft

Abhishek-Varma wants to merge 1 commit into nod-ai:main from Abhishek-Varma:llama65b_v1

Contributor

Abhishek-Varma commented Aug 16, 2023

Adding a WIP patch over @dan-garvey 's patch here for int4/fp32.

Apart from the bug fixes, I have also changed the order of compilation (Second and then First Vicuna/Llama).

Have put it on compilation - if IR gets generated we should be able to use the diff of this patch on top of @dan-garvey 's patch.


          Llama65B patch for int4 fp32

bd5b2b4

Signed-off-by: Abhishek Varma <[email protected]>

Abhishek-Varma requested a review from dan-garvey

August 16, 2023 17:28

Contributor Author

Abhishek-Varma commented Aug 16, 2023

Would need to get rid of the hardcoded dumps of IR post compilation's success before getting the diff of this patch in.
CC: @dan-garvey

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet